1. Field of the Invention
This invention relates to methods and apparatus for encoding and decoding multimedia data such as video data or audio data with error management.
2. Description of the Related Art
Widespread use of the Internet and wireless communication has increased demand for multimedia services that stream media over the Internet and mobile/wireless channels. In Internet Protocol (IP) networks, video can be provided by a server and can be streamed by one or more wired or wireless clients. Wired connections include dial-up, integrated services digital network (ISDN), cable, digital subscriber line protocols (collectively referred to as xDSL), fiber, local area networks (LAN), wide area networks (WAN) and others. Electronic devices utilizing wireless communications include telephones (e.g., cell phones), personal data assistants (PDAs), hand-held and portable computers and others. In most, if not all of these applications, bandwidth requirements and/or restrictions necessitate that video processing utilize a source encoder incorporating video compression algorithms to analyze, quantify and represent video data to convey the maximum information by expending a “minimum” number of bits. Characteristics of video processing using compressions algorithms can vary significantly based on content, which can lead to large scale variations in their performance (such as compression efficiency and bit rate).
Wireless channels are prone to errors. A part of transmitted data is “erroneous” if it is lost or corrupt. Because video compression inherently removes redundancy, the compressed data becomes critical. Any data that becomes erroneous during transmission may impact reconstructed video quality at the decoder. The impact is aggravated if the erroneous data is part of the reference frame for motion compensated prediction, causing temporal error propagation.
To alleviate impact on reconstructed video quality due to erroneously received bitstream data, the video decoder processes the received video data to improve video quality. This is referred to as error concealment. Error concealment schemes make use of the spatial and temporal correlation that exists in the video signal. Accordingly, these schemes are called spatial or temporal error concealment.
Depending on the nature of the channel errors (bit errors and/or packet loss) and the type of packetization, all or part of the data pertaining to one or more macroblocks or slices could be erroneous during transmission. For a macroblock, various coded macroblock data could be erroneous, which includes one or more of the following: coding mode (inter or intra), motion vectors (if inter), and transform coefficients.
If a macroblock is erroneous, it is typically assumed that all of the above information is erroneous. For an inter-coded macroblock, its motion vectors and residual coefficients are erroneous. Since residual signals are highly de-correlated, they are hard to estimate from neighboring macroblocks. However, studies have shown that reconstructed video quality may be greatly improved if motion vectors can be recovered. Hence, the main goal of temporal error concealment is to estimate the erroneous motion vectors from its spatial and temporal neighbors. Once the motion vector is constructed, motion compensation can then be performed using the constructed motion vector with zero residual coefficients.
Although there are several existing temporal error concealment schemes, they fail to offer good results. Moreover, they are not compatible with bi-directionally predicted multimedia data and/or variable motion compensation block sizes. Therefore, a new temporal error concealment scheme is needed to achieve improved results and provide wider compatibilities.